A Probability-Based Approach to Soft Discretization for Bayesian Networks

This report discusses how soft discretization can be implemented to train a discrete Bayesian Network directly from continuous data. The method consists of a soft discretization step that converts the continuous variables of the training cases into soft evidence, followed by a suitable parameter learning algorithm for the Bayesian Network. The learning algorithm is a modification of the Maximum Likelihood Estimation algorithm which is modified to accept soft evidence as input. We also discuss how to use soft discretization for inference and how to convert the inference results from the discrete network to meaningful continuous output values. Most literature on the use of soft discretization for Bayesian Networks proposes to use fuzzy set theory which is based on membership functions. Our approach goes back one step further and starts out with a probability density function that spreads the influence of a continuous variable to its neighbors, followed by a discretization step. Thus our approach to soft discretization is based on probability theory, rather than fuzzy set theory. We then show an interesting connection between these approaches. Namely, a membership function can be generated from the probability density function through convolution, yielding a set of probability-based membership functions. Prime applications of this method include any system with limited training data whose underlying mechanism is continuous in nature. These types of applications are common in the natural sciences and medicine. Using the continuity of the system, i.e. the fact that neighboring states in a continuous system are related to each other, we hope that soft discretization can yield more robust and more accurate models from small sample sizes. This report describes the method in enough detail to allow anyone to implement it themselves. Preliminary tests indicate increased robustness, but extensive tests of the performance of the new models in comparison to traditional models have yet to be performed.